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Abstract 

Photogrammetrics,  the  science  of  extracting  quantitative 
and  qualitative  data  from  multiple  sequential  recorded 
images,  has  been  an  integral  part  of  flight  test  and  evalua¬ 
tion  at  the  Naval  Air  Warfare  Center  Aircraft  Division 
(NAWCAD)  at  Patuxent  River,  MD  for  over  35  years. 
Photogrammetrics  analysis  is  used  for  evaluation  of 
stores  separation,  carrier  suitability,  range  tracking,  over¬ 
head  impact  scoring,  and  mishap  reconstruction.  The 
NAWCAD  photogrammetrics  team  is  pursuing  strategies 
to  reduce  the  time  and  increase  the  accuracies  of  solution 
processes  that  historically  have  been  labor-  intensive, 
prone  to  repetition  of  effort,  and  difficult  to  present.  This 
paper  describes  how  NAWCAD  scientists  have  applied 
a  clearly  defined  process  for  photogrammetrics  efforts, 
and  have  implemented  state-of-the-art  hardware  and 
software  methodologies  that  reduce  the  tumaraound 
time,  increase  the  accuracy,  and  facilitate  the  delivery  of 
[  custom  formatted  products  to  the  flight  test  engineer’s 


Overview 

Photogrammetrics  is  the  science  of  extracting  quantita¬ 
tive  and  qualitative  data  from  multiple  sequential  re¬ 
corded  images.  Derived  from  film  and  video, 

_ photogrammetrics  data  is  analyzed  to  assist  in  deter- 

,  mining  the  attitudes  and  position  of  objects  (recorded  in 
a  two-dimensional  image)  in  three-dimensional  space. 
;.f;  Jhotogrammetrics  applications  at  the  Naval  Air 
S',';'  Warfare  Center  Aircraft  Division  (NAWCAD)  include: 

*  Carrier  Suitability 

i 

.  ®  .t  •  Stores  Separation 

\  *  c 

,  w  "  •  Range  Tracking 

jj  “  «  •  Overhead  Impact 

•  Mishap  Reconstruction 

“  |jl  hotogrammetrics  events  require  a  great  deal  of  flexi- 
”•2  Ability  in  the  design  and  application  of  solution  algo- 
- nthras.  Photogrammetrics  events  occur  in  environ¬ 
ments  that  can  be  hostile  to  a  science  rooted  in  precise 


94 


22 


measurements.  In  designing  solution  algorithms, 
photogrammetrics  experts  must  take  into  account  fac¬ 
tors  such  as  camera  angle,  camera  movement,  curvature 
of  the  earth,  film  quality,  lens  focal  length  and  curva¬ 
ture,  and  environmental  conditions.  In  the  case  of  stores 
separation,  cameras  are  attached  to  a  moving  aircraft 
and  configured  to  capture  the  descent  of  a  store  or  multi¬ 
ple  stores  attached  to  that  aircraft.  The  photogrammet¬ 
rics  team  has  developed  solution  algorithms  to  accom¬ 
modate  a  variety  of  conditions,  including  camera 
malfunction,  camera  movement,  missing  IR1G  time  on 
film,  and  other  conditions  which  may  affect  the  accura¬ 
cy  of  the  photogrammetrics  solution. 

The  NAWCAD  photogrammetrics  team  has  developed 
a  photogrammetrics  process  consisting  of  seven  phases: 

1.  Develop  Test  Plan 

2.  Mark  and  Survey 

3.  Camera  Orientation 

4.  Mission 

5.  Film  Processing 

6.  Data  Reduction 

7.  Error  Analysis,  Answers,  Graphics 

The  success  of  each  phase  of  the  photogrammetrics  pro¬ 
cess  is  dependent  upon  the  integrity  of  the  preceding 
phasc(s).  For  example,  solution  algorithms  used  during 
the  data  reduction  phase  rely  on  the  accuracy  of  mea¬ 
surements  obtained  during  the  mark  and  survey  phase. 
A  wide  variety  of  variables  can  affect  the  outcome  of  the 
photogrammetrics  analysis.  The  photogrammetrics 
team  has  developed  standard  operating  procedures  for 
each  phase  of  the  photogrammetrics  process.  While  in¬ 
dividual  projects  often  present  unique  challenges,  the 
photogrammetrics  team  maintains  the  integrity  of  the 
photogrammetrics  solution  by  approaching  each  project 
from  a  proven  methodology  platform.  The  photogram¬ 
metrics  configuration  must  generate  the  flexibility  to 
adapt  to  a  wide  range  of  conditions  while  adhering  to  an 
operating  methodology  that  promotes  a  cohesive  team- 
oriented  strategy  for  obtaining  accurate  photogrammet¬ 
rics  solutions. 
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Applications 

Carrier  Suitability  &  Range  Tracking 

Carrier  suitability  and  store  separation  projects  form  the 
bulk  of  photogrammetrics  work  at  NAWCAD.  Carrier 
suitability  events  are  flown  on  specific  operational  U.S. 
Navy  aircraft  carriers  or  at  the  Mark  7  arresting  site  and 
the  TC-7  Catapult  site  at  NAWCAD,  Patuxent  River, 
Maryland.  Cameras  are  mounted  at  the  site  of  the  arrest¬ 
ment  or  take-off  (see  Figure  1). 


Figure  1.  Mark-7  arresting  gear  camera  configuration. 

For  carrier  suitability  applications,  photogrammetrics 
analysis  provides  aircraft  position  and  attitude  informa¬ 
tion  for  catapult  launches,  touch-and-go’s,  and  arrested 
landings  (see  Figure  2).  The  NAWCAD  Photogrammet¬ 
rics  Team  reduces  data  from  tests  performed  at  the 
NAWCAD  arresting/catapult  site,  carrier  certification 
trips,  and  other  NAWCAD  facilities. 
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Stores  Separation 

Photogrammetrics  analysis  provides  the  position  and 
orientation  of  a  store  with  respect  to  the  aircraft  during 
store  separation  (see  Figure  3).  Velocities  and  rates  are 
also  provided  using  smoothing  techniques.  To  record 
store  separation  events,  cameras  are  mounted  directly 
on  the  aircraft  In  addition  to  camera  angle,  camera 
movement  and  other  factors  indigenous  to  the 
photogrammetrics  environment,  the  camera  configura¬ 
tion  must  often  overcome  obstacles  such  as  additional 
instrumentation,  fuel  tanks,  vapor  trails,  and  other 
stores  which  may  block  one  or  more  camera  views  of  a 
portion  of  the  event 
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Figure  3.  Store  separation  event 

As  opposed  to  carrier  suitability  events  where  the  cam¬ 
eras  are  mounted  on  a  permanent  static  platform  at  the 
site  of  the  event,  store  separation  algorithms  confront 
truth  data  that  is  obtained  from  cameras  attached  to  the 
rapidly  moving  platform  (the  aircraft,  see  Figure  4),  it¬ 
self  the  primary  subject  of  the  photogrammetrics  event. 
In  addition,  the  cameras  themselves  are  subject  to 
movement  relative  to  the  store  due  to  wing  movement 
or  fluctuation  in  air  pressure/resistance. 


1F-1  Forward  (1FWD) 
2F- 2  Forward  (2FWO) 
1A-1AFT 
2A-2ATT 


•I— •  Foraud  (tFWD) 
7F- 7  Forward  (7FWS) 
tA-OAFT 
7 A- 7 AFT 


Figure  2.  Carrier  arrestment  (photogrammetrics  data 
points  superimposed ) 


Figure  4,  F-  14D  PDU  camera  configuration 
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Figure  5.  Photogrammetrics  phases 

Test  Plan  Development 

The  fust  Phase  in  the  photogrammetrics  process 
involves  developing  the  test  plan.  .  F^ch  will 

photogrammetrics  team  gathers  infotmanon  which 

guide  the  entire  photogrammetnes  process. 

The  photogrammetrics  team  must  ascemin  such  infer- 
mation  as: 

•  The  proposed  number  of  flights  per  day 

•  The  proposed  number  of  events  per  flight 

•  The  exact  nature  of  each  event  (single  or 
multiple  release) 

•  Expected  turnaround  time  for  delivery  of 
photogrammetrics  data 

.  The  number  of  cameras  to  be  attached  to  the 

aircraft 

•  The  number  and  size  of  the  stores 

•  Anv  additional  equipment  which  may  be 
Cached  to  the  aircraft:  specialized  instrumen¬ 
tation  packages,  fuel  tanks,  etc. 


•  Any  project -specific  information 

Of  special  interest  to  the  photogrammetrics  team  is  the 
design  of  the  targets  attached  to  or  painted  on  the  aircraft 
and  store.  Targets  are  placed  on  the  aircraft  and  store  to 
enable  the  photogrammetrics  team  to  identify  the  atti¬ 
tude  and  position  of  a  camera  (relative  to  the  store)  or 
a  released  store.  The  shape  and  color  of  the  target  are 
critical  to  the  accuracy  of  the  photogrammetrics  solu¬ 
tion.  If  the  target  is  clearly  identifiable  on  film  or  video, 
the  photogrammetrics  team  can  produce  accurate, 
timely  photogrammetrics  solutions.  A  poorly  designed 
target  will  hamper  the  photogrammetrics  process  and 
can  diminish  the  integrity  of  the  photogrammetrics 
solution. 

A  well  constructed  test  plan  enables  the  photogrammet¬ 
rics  team  to  make  the  preparations  necessary  to  obtain 
solutions  on  time  and  within  budget.  Certain  project- 
specific  conditions  may  require  software  modifications. 
Other  test  plan  specifications  may  require  upgrades  to 
the  existing  hardware  configuration.  A  strategic 
approach  to  the  test  planning  phase  will  help  to  ensure 
the  timely  and  accurate  delivery  of  photogrammetrics 
data. 

Aircraft  and  Stores  Survey 

After  the  test  plan  has  been  developed,  the  photogram¬ 
metrics  survey  team  performs  a  survey  of  the  aircraft 
and  stores.  Photogrammetrics  solution  algorithms  rely 
on  truth  data  obtained  from  the  initial  survey  of  the  air¬ 
craft  and  stores;  it  is  imperative  that  these  measure¬ 
ments  be  accurate.  Because  survey  tools  and  methods 
vary  and  because  photogrammetrics-oriented  surveys 
emphasize  measurements  that  may  have  little  signifi¬ 
cance  in  other  applications,  the  NAWCAD  photogram¬ 
metrics  team  employs  its  own  aircraft  and  stores  survey 
expert.  By  standardizing  the  tools  and  methods  for  air¬ 
craft  and  stores  surveys,  the  potential  for  erroneous,  in¬ 
compatible,  or  incomplete  survey  measurements  is  sig¬ 
nificantly  reduced.  A  meticulous, 
photogrammetrics-oriented  survey  provides  a  full 
range  of  measurements  that  can  be  applied  to  a  variety 
of  situations.  Accurate  and  complete  photogrammetrics 
survey  measurements  provide  an  increased  range  of 
options  when  problems  are  encountered  during  a 
photogrammetrics  event  or  analysis. 

Measurements  obtained  from  the  survey  of  aircraf  t  and 
stores  produce  data  that  is  crucial  to  photogrammetrics 
solutions.  The  survey  provides  the  position  of  targets 
located  on  a  •’♦ore.  In  addition,  the  survey  must  provide 
the  exact  lo<  n  of  cameras  attached  to  the  aircraft,  the 
distance  from  one  camera  to  another,  and  the  distance 


from  a  particular  camera  to  the  stowed  position  of  the 
store.  These  measurements  are  essential  to  the  accuracy 
of  photogrammetrics  solutions,  such  as  triangulation. 

Bach  type  of  photogrammetrics  application,  from  carrier 
suitability  to  overhead  impact,  entails  a  different  set  of 
challenges  and  requirements  for  photogrammetrics-ori- 
ented  surveys.  Within  each  photogrammetrics  applica¬ 
tion,  project-specific  requirements  often  produce 
unique  conditions  for  photogrammetrics  surveys. 
Regardless  of  the  specific  application,  the  photogram¬ 
metrics  survey  team  must  deliver  accurate  results  and 
maintain  the  flexibility  necessary  to  accommodate  a 
variety  of  situations. 

Camera  Orientation 

The  third  phase  of  the  photogrammetrics  process  deals 
with  camera  orientation.  During  the  test  planning 
phase,  the  photogrammetrics  customer  details  the  flight 
test  event  and  the  portion  of  the  event  that  should  be  cap¬ 
tured  for  photogrammetrics  analysis.  The  cameras  are 
then  configured  to  capture  the  specified  range  of  the 
event.  If  critical  portions  of  the  event  are  not  captured, 
additional  flights  may  be  required,  resulting  in  unnec¬ 
essary  time  delays  and  cost  overruns. 

Camera  orientation  is  based  on  several  factors: 

1.  View  volume  of  each  camera,  which  is  dic¬ 
tated  by  the  size  and  calibration  of  the  lens, 
the  type  of  film  used,  and  the  distance  from 
the  camera  to  the  object  being  viewed. 

2.  Instrumentation  of  the  camera  or  the  number 
of  frames  captured  per  second. 

3.  Number  of  cameras  used  and  the  location  of 
each  camera  (derived  from  the  initial  survey). 

4.  Data  obtained  during  the  test  plan  phase  such 
as  the  release  points  and  conditions  for  ejec¬ 
tion  of  store. 

Mission 

The  mission  phase  of  the  photogrammetrics  process 
consists  of  the  actual  flight  tests  and  is  under  the  control 
of  the  photogrammetrics  customer. 

As  is  the  case  in  every  phase  of  the  photogrammetrics 
process,  communication  between  the  photogrammet¬ 
rics  team  and  the  photogrammetrics  customer  is  vital.  If 
any  flight  test  parameters  are  changed  during  the  mis¬ 
sion,  the  photogrammetrics  team  must  be  informed. 
Difficulties  experienced  by  the  photogrammetrics  data 
reduction  team  must  be  communicated  immediately  to 
the  customer  so  that  ongoing  missions  can  be  modified. 

it,  during  tile  mission  phase,  the  release  points  are 
changed,  the  cameras  must  be  reoriented. 


If  the  test  plan  has  been  adequately  developed,  the  sur¬ 
vey  accurately  performed,  and  the  cameras  properly  ori¬ 
ented,  then  the  mission  should  result  in  cost-effective, 
accurate  solutions. 

Film  Processing 

Film  processing  is  performed  by  the  NAWCAD  Photo 
Lab.  Once  again,  communication  plays  a  key  role.  If  a 
high  volume  of  film  is  to  be  processed,  the  photo  lab 
must  be  given  enough  lead  time  to  prepare  for  increased 
processing.  During  the  test  plan  development  phase,  an 
estimate  should  be  given  for  film  processing  volume 
and  expected  turnaround  time.  If  the  project  uses  video 
instead  of  film,  then  the  film  processing  phase  is  elimi¬ 
nated  and  the  photogrammetrics  team  can  immediately 
proceed  to  the  data  reduction  phase. 

Data  Reduction 

The  sixth  phase  of  the  photogrammetrics  process,  data 
reduction,  represents  the  most  time-consuming,  labor- 
intensive  portion  of  the  photogrammetrics  process. 
Data  reduction  involves  two  steps:  data  extraction  and 
solution  calculation.  Data  extraction  is  performed  using 
two  primary  systems:  the  Telereadex  film  reading 
machines  and  the  Semi-Automated  FilmJVideo  Reader 
(SAFVR).  Once  extracted,  the  data  is  fed  into  one  or 
more  data  reduction  software  packages .  The  data  reduc¬ 
tion  software  converts  the  data  to  the  desired  format, 
performs  various  photogrammetrics  solution  algo¬ 
rithms,  and  provides  quality  assurance  verification. 

Data  Extraction 

The  bulk  of  data  extraction  work  is  performed  on  the 
Telereadex  film  reading  machines.  Data  from  film  or 
video  must  be  extracted  and  converted  to  a  format  that 
can  be  read  by  a  variety  of  data  reduction  and  software 
analysis  packages.  The  Telereadex  machines  are  a 
proven  tool  for  obtaining  photogrammetrics  data  from 
film. 

Each  film  frame  is  encoded  with  numbers  that  corre¬ 
spond  to  specific  events.  After  the  film  is  loaded  onto 
the  Telereadex,  the  film  is  fast  forwarded  to  the  first 
frame  in  the  first  event  to  be  read.  Film  reading  experts 
align  vertical  and  horizontal  (x,y)  cross  hairs  on  speci¬ 
fied  points  on  the  object  to  be  analyzed.  The  cross  hairs 
are  linked  tc  a  numeric  mapping  system  which  records 
thex.y  location  of  each  point  read.  For  store  separation, 
the  points  read  axe  potiioiis  of  targets  attached  to  the 
store  and  the  aircraft  The  number  of  points  read  are 
determined  by  the  target  design,  the  shape  and  size  of 


the  store,  the  visibility  of  the  store  and  targets,  and  other 
event-specific  factors. 

The  data  extraction  process  is  similar  for  the  Semi- Au¬ 
tomated  Film/Video  Reader  (SAFVR)  —  except  that 
mice  the  system  has  been  setup  to  read  a  specified  event, 
the  points  ate  read  automatically  in  an  ideal  scenario; 
experience  with  the  SAFVR  technology  has  been  that 
manual  intervention  and  reading  assistance  is  often  re¬ 
quired.  The  SAFVR  system  reduces  video  as  well  as 
film.  SAFVR  is  a  one-of-a-kind  system  procured  and 
developed  specifically  for  photogrammetrics  data 
reduction  at  NAWCAD. 

The  success  or  failure  of  the  data  extraction  phase  is 
linked  to  the  integrity  of  the  previous  phases  of  the 
photograrumetrics  process.  The  survey  of  the  aircraft 
and  store  is  performed  using  data  from  die  test  plan.  The 
camera  orientation  is  accomplished  using  data  from  the 
test  plan  as  well  as  measurements  obtained  from  the 
survey.  If  the  communication  process  is  maintained  dur¬ 
ing  the  mission  and  film  processing  phases,  then  the 
required  data  will  be  captured  on  film  and  extracted  for 
data  reduction  purposes. 

Factors  such  as  film  quality,  weather,  mission  time  of 
day,  and  camera  malfunctions  can  affect  the  data  extrac¬ 
tion  process.  The  photograinnetrics  team  must  main¬ 
tain  the  flexibility  to  adjust  to  conditions  beyond  its 
control.  As  long  as  the  integrity  of  the  photogrammet- 
rics  process  is  not  compromised,  the  photogrammetrics 
team  can  often  successfully  respond  to  unforeseen  cir¬ 
cumstances.  If.  for  example,  a  particular  camera  breaks, 
the  data  reduction  algorithm  can  be  modified  to  account 
for  missing  data.  If  the  remaining  data  is  accurate,  the 
photogrammetrics  solution  will  not  suffer  a  significant 
reduction  in  accuracy. 

Data  Reduction  Solution  Algorithms 

The  NAWCAD  photogrammetrics  team  has  developed 
photogiammetrics  software  packages  to  support  a  wide 
range  of  requirements.  Our  primary  requirement  is  to 
ensure  that  the  data  is  accurate.  Once  extracted  from 
film  or  video,  the  data  is  run  through  one  or  more  data 
validation  software  packages.  After  it  has  been  vali¬ 
dated,  the  data  is  run  through  a  software  package  geared 
towards  specific  customer  requirements. 

Typically,  store  separation  flight  testing  consists  of 
numerous  flights  that  approach  the  edge  of  the  separa¬ 
tion  envelope  in  small,  incremental  steps.  Store  separa¬ 
tion  analysis  must  define  the  separation  envelope.  The 
reparation  envelope  encompasses  the  range  of  parame¬ 
ters  that  indicate  that  conditions  arc  safe  for  ejection  of 
the  store.  The  edge  of  of  the  separation  envelope  is  the 


final  point  at  which  a  store  can  be  safely  ejected  from  an 
aircraft.  The  standard  single-camera  photogrammetrics 
solution  is  easy  to  use  and  provides  reasonable  results. 
However,  the  required  number  of  flight  test  events  make 
flight  testing  costs  prohibitive. 

To  assist  in  reducing  the  amount  of  flight  testing  and 
increase  the  accuracy  of  the  results,  the  NAWCAD 
photogrammetrics  team  developed  the  multicamera 
triangulation  algorithm.  Three  or  more  cameras  are  used 
to  quantify  error.  Four  or  more  cameras  are  used  to 
quantify  relative  error.  To  achieve  a  solution, 
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T(x,y,z)  =  ^t(,»a2 
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is  minimized  by  finding  x,y,  and  z,  satisfying: 


VT(x,y,z)  *  (0,0,0) 


Multicamera  triangulation  enables  photogrammetrics 
customers  to  approach  the  edge  of  the  separation  enve¬ 
lope  with  a  greater  degree  of  confidence. 

When  triangulation  can't  be  accomplished,  multicam¬ 
era  projection  is  applied.  With  multicamera  projection, 
a  target  remains  useful  even  if  it  is  seen  by  only  one  cam¬ 
era.  The  predicted  x,y,z  of  a  target  can  be  projected  onto 
any  film  plane. 

The  photogrammetrics  team  has  developed  software 
packages  for  the  full  range  of  photogrammetrics 
applications  from  carrier  suitability  to  mishap  recon¬ 
struction. 

Product  Delivery 

To  complete  the  photogrammetrics  process,  the 
photogrammetrics  team  must  deliver  photogrammet¬ 
rics  solutions  in  the  formats  specified  by  the  customer, 
when  and  where  the  customer  needs  them.  Often,  solu¬ 
tions  are  delivered  in  the  form  of  graphs  or  tabular  lists. 
To  allow  customers  to  perform  additional  analysis  at 
their  own  desktops,  photogrammetrics  data  is  converted 
to  generic  formats  that  can  be  processed  by  most  major 
software  analysis  packages. 
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Robert  Standi  is  the  photogrammetrics  project  coordinator  for  the  Software  and 
Engineering  Applications  Department  (SEAD)  of  the  Range  Directorate  (RD)  at  the  Naval 
Air  Warfare  Center  Aircraft  Division  (NAWCAD),  Patuxent  River,  MD.  A  computer  scien¬ 
tist  for  SEAD  since  19S9,  Mr.  Stancil 's  previous  assignments  have  included  the  design  and 
development  of  graphics  software  and  numerous  flight  test  data  reduction  programs  for  pro¬ 
jects  including  the  A-6E  Rewing  and  various  T-45,  F-14,  and  F-18  programs.  In  March 
1993,  Mr.  Stancil  joined  the  Photogrammetrics  group  in  SEAD,  where  he  assisted  in  flight 
test  analysis  for  the  F-14  Pre-Deployment  Update  (PDU)  project.  In  November  *93,  he  was 
designated  SEAD’s  single  point-of-contact  for  organizing  and  coordinating  film  and  video 
projects  requiring  photogrammetrics  analysis  hrom  the  Strike  Aircraft  Test  Directorate, 
Rotary  Wing  Directorate,  and  Force  Warfare  Directorate  at  NAWCAD,  Patuxent  River,  MD. 
Mr.  Stancil  was  graduated  in  1988  from  St  Mary's  College  of  Maryland  with  a  Bachelor  of 
Science  degree  in  mathematics  and  computer  science. 

Mr.  Alec  E.  (Ed)  Forsman  is  the  primary  software  developer  for  NAWCAD  photogram¬ 
metrics  applications.  For  the  past  seven  years,  as  a  computer  scientist  working  for 
NAWCAD’s  Range  Directorate,  Mr.  Foreman's  efforts  have  involved  many  post-flight  data 
reduction  tasks  concerning  almost  all  types  of  U.S.  Navy  aircraft  He  has  developed  many 
of  the  photogrammetric  applications  currently  in  use  at  NAWCAD.  Mr  Foreman  coauthored 
the  paper  '‘Photogrammetric  Methods  on  a  Cold  Weather  Range,”  presented  to  the  Range 
Commanders  Council  in  1990.  Mr.  Foreman  is  a  graduate  of  the  University  of  Chicago, 
where  he  received  a  bachelor’s  degree  in  mathematics  in  1987. 
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